Support timeToLive attributes in DynamoDb Enhanced Client#6152
Support timeToLive attributes in DynamoDb Enhanced Client#6152roamariei wants to merge 2 commits intoaws:masterfrom
Conversation
28f6637 to
5206b59
Compare
9b7a52a to
420e193
Compare
97468cb to
2291714
Compare
2291714 to
207b147
Compare
7673952 to
fa8d740
Compare
| public String baseField; | ||
| public long duration; | ||
| public ChronoUnit unit; |
There was a problem hiding this comment.
Should these be public or private ?
There was a problem hiding this comment.
I think these should be private and final to easily ensure the @ThreadSafe is respected.
There was a problem hiding this comment.
Agree, updated from public to private final.
| @ThreadSafe | ||
| public final class TimeToLiveExtension implements DynamoDbEnhancedClientExtension { | ||
|
|
||
| public static final String CUSTOM_METADATA_KEY = "TimeToLiveExtension:TimeToLiveAttribute"; |
There was a problem hiding this comment.
1- Consider renaming this to something like, TTL_ATTRIBUTE_METADATA_KEY, current name is too vague
2- This Key is under @SdkPublicApi class, because the class is @SdkPublicApi, everything public on it becomes part of the public API contract, but it is only used from UpdateTimeToLiveOperation which is an SdkInternalAPI for any backward compatibility issues going forward. Consider either making the field package-private (so UpdateTimeToLiveOperation in the same package can still access it), or moving the constant to an @SdkInternalApi shared utility class...
There was a problem hiding this comment.
- Agree, but this is the pattern used for the other extensions.
- I would go with the last approach, and create an @SdkInternalApi shared utility class - and place there all *_ATTRIBUTE_METADATA_KEY for extensions. Would make sense? wouldn't make too much sense to create the class only for TimeToLive extension attributes... what do you think?
| public Consumer<StaticTableMetadata.Builder> modifyMetadata(String attributeName, | ||
| AttributeValueType attributeValueType) { | ||
| Map<String, Object> customMetadataMap = new HashMap<>(); | ||
| customMetadataMap.put("attributeName", attributeName); | ||
| customMetadataMap.put("baseField", baseField); | ||
| customMetadataMap.put("duration", duration); | ||
| customMetadataMap.put("unit", unit); | ||
|
|
||
| return metadata -> metadata.addCustomMetadataObject(CUSTOM_METADATA_KEY, | ||
| Collections.unmodifiableMap(customMetadataMap)); |
There was a problem hiding this comment.
BeanTableSchema validates that only one TTL attribute exists, but StaticTableSchema has no such guard. Since modifyMetadata stores a Map (not a Collection) via addCustomMetadataObject, a second TTL attribute silently replaces the first with no error:
StaticTableSchema.builder(MyItem.class)
.addAttribute(Long.class, a -> a.name("ttl1")
.getter(MyItem::getTTL1).setter(MyItem::setTTL1)
.tags(timeToLiveAttribute("createdAt", 30, ...)))
.addAttribute(Long.class, a -> a.name("ttl2")
.getter(MyItem::getTTL2).setter(MyItem::setTTL2)
.tags(timeToLiveAttribute("updatedAt", 7, ...)))
.build();
// ttl1 config is silently lost, only ttl2 is trackedDynamoDB only supports one TTL attribute per table, so this is always a user error, but the SDK should fail loudly rather than silently dropping one.
Recommendation: You may consider moving the validation into TimeToLiveAttribute.modifyMetadata() so it works for all schema types
There was a problem hiding this comment.
Agree, the validation is applied only on BeanTableSchema and I am preparing parameterized tests to cover the remaining schemas with validation updated - tests still in progress, have pretty many scenarios to cover with different record types.
Also, I feel like addCustomMetadataObject is not consistent in terms of validation:
private void mergeCustomMetaDataObject(String key, Object object) {
if (object instanceof Collection) {
this.addCustomMetadataObject(key, (Collection<Object>) object);
} else if (object instanceof Map) {
this.addCustomMetadataObject(key, (Map<Object, Object>) object);
} else {
this.addCustomMetadataObject(key, object);
}
}because the else branch throws an exception when finds duplicates:
public Builder addCustomMetadataObject(String key, Object object) {
if (customMetadata.containsKey(key)) {
throw new IllegalArgumentException("Attempt to set a custom metadata object that has already been set. "
+ "Custom metadata object key: " + key);
}
customMetadata.put(key, object);
return this;
}| TimeToLiveExtension.builder().build(); | ||
| private static final List<DynamoDbEnhancedClientExtension> DEFAULT_EXTENSIONS = | ||
| Arrays.asList(DEFAULT_VERSIONED_RECORD_EXTENSION, DEFAULT_ATOMIC_COUNTER_EXTENSION); | ||
| Arrays.asList(DEFAULT_VERSIONED_RECORD_EXTENSION, DEFAULT_ATOMIC_COUNTER_EXTENSION, DEFAULT_TIME_TO_LIVE_EXTENSION); |
There was a problem hiding this comment.
This means every existing DynamoDbEnhancedClient using default extensions will now run TimeToLiveExtension.beforeWrite() on every write operation, even for tables
with no TTL attributes. The extension does an early return when no metadata is found, but we still change the behavior:
-
Performance: An additional extension in the chain for every write across the entire SDK, for all users, even those who never use TTL. The customMetadataObject() lookup + Optional.orElse(null) runs on every putItem/updateItem/batchWriteItem/transactWriteItems.
-
Double registration: Users who extend the defaults will register TTL twice:
// This is how TimeToLiveRecordTest in this PR does it — TTL extension runs twice per write
DynamoDbEnhancedClient.builder()
.extensions(Stream.concat(
ExtensionResolver.defaultExtensions().stream(),
Stream.of(timeToLiveExtension))
.collect(Collectors.toList()))
.build();Should we think about making this extension opt-in instead ?
also, if we want to keep this, The Javadoc comment on defaultExtensions() says "Currently this is just the VersionedRecordExtension", we may need to update it
| default DescribeTableEnhancedResponse describeTable() { | ||
| throw new UnsupportedOperationException(); | ||
| } | ||
|
|
There was a problem hiding this comment.
consider adding javadoc, also applies to UpdateTimeToLiveEnhancedResponse and to Async table
There was a problem hiding this comment.
javadoc added
| if (baseValue instanceof LocalTime) { | ||
| return LocalDate.now().atTime((LocalTime) baseValue).plus(duration, unit).toEpochSecond(ZoneOffset.UTC); | ||
| } |
There was a problem hiding this comment.
Do we want to support local time ?
if (baseValue instanceof LocalTime) {
return LocalDate.now().atTime((LocalTime) baseValue).plus(duration, unit).toEpochSecond(ZoneOffset.UTC);
}two potential problems:
- Midnight jump: LocalDate.now() defines what "today" means, and that changes at midnight. Items with the same LocalTime base value and duration get TTLs that differ by a day depending on which side of midnight the write lands:
- Write at 11:59 PM March 10 -> LocalDate.now() = March 10-> TTL = March 11 00:05 (expires in ~6 minutes)
- Write at 12:01 AM March 11 -> LocalDate.now() = March 11-> TTL = March 12 00:05 (expires in ~24 hours)
- JVM timezone dependency: LocalDate.now() uses ZoneId.systemDefault(), but the epoch conversion uses ZoneOffset.UTC. If the JVM runs in America/New_York, LocalDate.now() could return a different date than UTC, shifting the TTL by hours depending on where the server runs.
If you say, LocalDate.now(ZoneOffset.UTC), it would fix problem 2, but problem 1 is still there to LocalTime having no date component, there's always a midnight boundary where the TTL jumps by 24 hours.
Recommendation: You may consider removiong LocalTime support and let it fall through to the existing IllegalArgumentException if that's acceptable. The other five types (Instant, LocalDate, LocalDateTime,
ZonedDateTime, Long) are all deterministic and cover every real TTL use case. Users who need time-of-day-based TTL can compute the epoch seconds themselves with explicit control over the date and timezone.
There was a problem hiding this comment.
I strongly suggest we only use UTC and only support Instant with a Clock parameter like in AutoGeneratedTimestampRecordExtension.
| String baseFieldName = (String) customTTLMetadata.get("baseField"); | ||
| Long duration = (Long) customTTLMetadata.get("duration"); | ||
| TemporalUnit unit = (TemporalUnit) customTTLMetadata.get("unit"); |
There was a problem hiding this comment.
If any key is missing or has a wrong type (like someone manually constructs metadata via StaticTableMetadata.builder().addCustomMetadataObject(...)), these lines will throw NullPointerException with no context about which key failed or why. Can consider to use Validate.notNull() with descriptive messages..
There was a problem hiding this comment.
I agree and a wrong type would throw a different exception and require a dedicated check.
There was a problem hiding this comment.
Added validation.
|
|
||
| private final boolean enabled; | ||
|
|
||
| public UpdateTimeToLiveOperation(boolean enabled) { |
There was a problem hiding this comment.
Is this supposed to be private ?
| * | ||
| * @return The properties of the timeToLive specification. | ||
| */ | ||
| public TimeToLiveSpecification table() { |
There was a problem hiding this comment.
table() function name is misleading, may consider renaming it to something like timeToLiveSpecification() ?
There was a problem hiding this comment.
Updated, should be fine now.
| import software.amazon.awssdk.utils.Validate; | ||
|
|
||
| /** | ||
| * Defines the elements returned by DynamoDB from a {@code DescribeTimeToLive} operation, such as |
There was a problem hiding this comment.
I guess not DescribeTimeToLive , probably mistake ?
There was a problem hiding this comment.
Yes, updated.
| import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbPartitionKey; | ||
|
|
||
| @DynamoDbBean | ||
| public class RecordWithDefaultTTL { |
There was a problem hiding this comment.
Is this and RecordWithSimpleTTL dead code?
| builder.attributes(attributes); | ||
|
|
||
| if (ttlAttributesCount.intValue() > 1) { | ||
| throw new IllegalArgumentException( |
There was a problem hiding this comment.
I think we should cover this in the unit tests.
|
Should we add TTL validation to |
|
|
||
| private TimeToLiveAttribute(String baseField, long duration, ChronoUnit unit) { | ||
| this.baseField = baseField; | ||
| this.duration = duration; |
There was a problem hiding this comment.
Should we check this is not negative somewhere?
There was a problem hiding this comment.
Added validation for duration in modifyMetadata and beforeWrite, I will have a look if there is a better place for this validation.
| if (customTTLMetadata == null) { | ||
| throw new IllegalArgumentException("Custom TTL metadata object is null"); | ||
| } | ||
| String ttlAttributeName = (String) customTTLMetadata.get("attributeName"); |
There was a problem hiding this comment.
Why are we not using validateMetadataValue helper method ?
| Map<String, ?> customTTLMetadata = tableSchema.tableMetadata() | ||
| .customMetadataObject(CUSTOM_METADATA_KEY, Map.class).orElse(null); | ||
| if (customTTLMetadata == null) { | ||
| throw new IllegalArgumentException("Custom TTL metadata object is null"); |
There was a problem hiding this comment.
I think this Message is Confusing a bit, it exposing Internal Logic, instead of helping caller to help fix the issue.
It should be more readable to caller having something like No TTL attribute is configured in the table schema for table, Use @DynamoDbTimeToLiveAttribute on a Long getter....
| @Retention(RetentionPolicy.RUNTIME) | ||
| @BeanTableSchemaAttributeTag(TimeToLiveAttributeTags.class) | ||
| @SdkPublicApi | ||
| public @interface DynamoDbTimeToLiveAttribute { |
There was a problem hiding this comment.
Classes / Interfaces should have @ since tags
| long duration = validateMetadataValue(customTTLMetadata, "duration", Long.class); | ||
| TemporalUnit unit = validateMetadataValue(customTTLMetadata, "unit", TemporalUnit.class); | ||
|
|
||
| Validate.isTrue(duration >= 0, "Custom TTL metadata key 'duration' must not be negative."); |
There was a problem hiding this comment.
I think we should add baseFieldName check as well, here.
What happens if User does @DynamoDbTimeToLiveAttribute(duration=30, unit=DAYS) — forgot baseField.
It would be silent no-op when duration > 0 and baseField="".

Description
Support TimeToLive attributes in DynamoDb Enhanced Client
Motivation and Context
#4295
Modifications
Added DynamoDbTimeToLiveAttribute annotation, TimeToLive extension to run before every write, added new operations DescribeTimeToLive and UpdateTimeToLive in enhanced client.
Testing
Added unit tests for TimeToLiveExtension, DescribeTimeToLiveOperation, UpdateTimeToLiveOperation and integration tests for update timeToLive with enhanced client in TimeToLiveRecordTest.
Test Coverage Checklist
Screenshots (if appropriate)
Types of changes
Checklist
mvn installsucceedsscripts/new-changescript and following the instructions. Commit the new file created by the script in.changes/next-releasewith your changes.License